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ABSTRACT 

In an examination of the meaning and measurement of 
quality in the undergraduate experienced quality is compared with the 
conceptually similar but distinct concepts of adequacy and 
excellence. Most of the conceptual frameworks available for assessing 
quality are essentially unidimensional assessment strategies that 
rely almost exclusively on quantitative indicators such as student 
ability or library resources. The most popular multidimensional 
approach to assessing quality is the input-environment-output model. 
A redefinition of Stuff lebeam' s et al. planning and evaluation model 
includes: context, input, involvement, and outcome. Using these 
categories, the opinion and empirical research related to quality 
were reviewed to assess indices of quality. Quantitative quality 
assessment usually focuses on institutional factors about which 
objective, standardized measures are available or can be collected, 
and produce data that (1) can be used for both intra- and 
interinstitutional comparisons; (2) are amenable to computer-assisted 
analytic procedures; and (3) are compatible with the psychometric 
paradigm. For qualitative quality assessments, manifestations such as 
students' reports of satisfaction with various aspects of the 
institution or observations of students' involvement in the classroom 
or other activities are primary data sources. Holistic quality 
assessments are concerned with: the purpose of the target programs, 
information about involvement and outcomes of college attendance, an 
action-orientation, multiple forms of data-gathering, a public and 
educative function, and a value orientation. (SW) 
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THE MEANING AND MEASUREMENT OF QUALITY 
IN THE UNDERGRADUATE EXPERIENCE 



Most institutions of hiaher education 'share a history of steady 
and, at times, almost exponential expansion. At present, a large 
proportion of colleges and universities are preparing to educate about 
the same or slightly smaller numbers of students with more diverse 
learning requirements but have fewer "real" (inflation adjusted) 
dollars available. Concurrently, societal expectations for education 
remian unreal istically high even though the "new Federalism" and sluggish 
local economies have resulted in reduced appropriation levels. For 

/ 

these as well as other context specific reasons colleges and universities 
have begun to look inward to determine how to maintain "quality". Indeed, 
in 1980 the two largest higher education associations used quality as the 
cornerstone of their respective convention themes. 

A number of researchers have used various methods to produce 
"reputational ratings" of institutional "quality" (e.g., Blau & Margulies, 
1975; Cartter, 1966; Roose & Anderson, 1970). For the most part, these 
ratings have focused cn department or major fields as the unit of analysis 
With few exceptions (e.q., Heath, 1968), quality of the undergraduate 
experience £er_ s£ has not been treated in any detail in the literature. 

Few disagree that quality is an appropriate goal to pursue. Often 
avoided or overlooked in the search for quality is v/hat the concept 
essentially entails. That is, the meaning of quality is rarely dis- 
cussed. Without considering v/hat quality implies, efforts to adequately 
asses quality lack focus. In this paper, the meaning and measurement 
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of quality in the undergraduate experience are examined. First, 
some consideration is given to what quality connotes. Quality is 
compared with the conceptually similar but distinct concepts of 
adequacy and excellence. Then, frameworks used to estimate quality 
in the undergraduate experience are reviewed and an alternative 
approach to aqnrenatinq indices of quality is described and use^ 
in b review of the literature on qualicy in higher education. The ad- 
vantages and disadvantages of various approaches to measuring quality 
are discussed. Finally, guiding principles for holistic quality assess- 
ments are presented. 

The Meaning of Quality 

The concept of quality has as many as eight different nuances 
(Merriam, 1976, p. 944). definitions of quality range from the vivid- 
ness of color to the timbre of a vowel sound. However, Straumanis (1931) 
has argued that the concept of quality is best represented by a judgment 
continuum anchored by somewnat different applications of the concept: 
(1) description; and (2) evaluation (see also Harshman, 1979). 

On the descriptive end, quality connotes a characteristic of 
someone or something wijjiout an appraisal of the value of the charac- 
teristic or the person or object exhibiting the characteristic (McCall, 
1976). For example, if a high school teacher attests that an applicant 
for admission to Siwash University "has personal qualities consistent 
with Siwash standards," the recommender probably means that the 
prospective student possesses certain traits and academic and social 
experiences that, taken together, suggest the student can be successful 
at Siwash. 
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In the evaluative meaning, quality is used to describe the value 
of an object,- person, or experience (Cartter, 1966), In the above 
example, if the^recommender wrote, "the* applicant^ has produced quality 
work throughout h^r four years at Eisenhower High School," the comment 
would suggest positive appraisal of the applicant's performance as con- 
trasted with a potentially negative evaluation of individual achievement. 

The quality continuum also reflects the degree to which clarity 
is represented in an object or experience (Campbell, Converse & Rogers, 
1976). Ind?5d., one of the reasons greater specificity has not been 
attempted in the literature is that quality is a rather ambiguous 
entity, particularly when applied to higher education. The W-e ambiguous 
an entity, the less agreement on what constitutes an entity. Yet, it 
is easier to agree on a description of an experience than an evaluative 
summary of the experience. The latter calls for declarations of 
values and personal taste which together require a clearer definition 
of the entity to be evaluated. Quality in the undergraduate experience 
is ambiguous and, therefore, somewhat easier to describe. But the 
value of college is much more difficult tc estinate because an evalua- 
tive assessment requires clarification of the variables and the re- 
lationships between variables that comprise students' experiences. 

Toward a Definition of Quality 

Quality is comprised of an elusive set of properties (Persig, 
1974). To produce quality, function is emphasized over form; i.e., 
the outcome or products of any given set of experiences are usually 
more important than the means used to produce the outcome. Quality 
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and cost may be positively correlated but the relationship is probably 
far from perfect (Jenny, 1979; Millett, 1979). For example, if an 
inexpensive item is satisfactory and satisfying such as a 29t Bic pen 
or a cut rate brand of aspirin, so much the better for the consumer. 
Similarly, a moderately priced institution, that meets the developmental 
needs and expectations of the student is probably preferable to a 
higher priced institution (Upton, 1963). ^ 

Quality' is not a normative property. That is, quality is\not 

\ 

well served by arbitrarily forming cateqories or classes of high, x 

\ 

medium, and low quality, and assigning proportionate numbers of 
academic programs or institutions to the various classes. Rather, ' x 
quality is determined by students and others applying standards to an 
experience during and after thoughtful consideration of the meaning 
of the myriad of interactions that comprised the student's experience. 

The effects of quality are expected to endure over a long term 
and positive (valued) if not similar outcomes are expected for most stu- 
dents most of tne time (see Olscamp, 1978). Quality infers a high degree 
9f fidelity between the purposes, philosophies, and goals of the 
institution and the behavior of persons frequenting the institution's 
environment (Checkering, 1969; Keeton, 1974). But the concept of 
quality also includes an esthetic property or some degree of sensuous 
pleasure. Quality may not necessarily reflect perfection, for educated 
persons without quirks and surprises tend to be rather uninteresting 
(Bowen, 1979). Indeed, such uniformity would be inconsistent with the 
aims of liberal education that appear in most college catalogues! 

Quality in the undergraduate student experience is better thought 
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of as a simple elegance, a perceived sense of well being and accomplish- 
ment (Witney, 1975) including a variety of sensations shared by partners 
in the enterprise. Students, faculty, •administrators, staff, and others 
feel comfortable and satisfied as a result of being ^volved in purpose- < 
ful experiences. However, not all constituencies, particularly but not 
exclusively students, may be comfortable or satisfied at all times during 
the undergraduate program. Curricular and cocurricular challenges that 
upset students' eguilibrium and encourage students to greater degrees of 
differentiation and personal integration are required to fulfill the 
development of the "whole person" goal to which most institutions subscribe 
(see Knefelkamp, 1980; Perry, 1970; Sanford, 1962). 

Adequacy, Excellence and Quality 

When applied to educational settings, the concepts of quality, 
adequacy, and excellence have similar but not interchangeable 
connotations. Therefore, unless distinctions are made between these 
terms, considerable confusion can result when interpreting the meaning 
of quality. 

All three concepts require comparison with some predetermined 
criteria to reflect attainment of a particular purpose or requirement. 
The shades of distinction between these terms can be illuminated 'by 
applying the criteria of merit and worth. Lincoln and Guba (1979) have 
argued that merit represents an intrinsic, context-free value, "inde- 
pendent of any requirements of applicability or use" (p. 1). Scientific 
discoveries are often meritorious because scientists appreciate an impor- 
tant addition to knowledge for its own sake. If a discovery is without 
theoretical significance (lacks merit) but has utility in a 'practical 
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context, the 1 discovery would be worthwhile. Worth, therefore, is an 
extrinsic, context specific value. 

Adequacy suggests a level of sufficiency for certain persons in 
a specific context or setting (McCarthy, 1981; Merriam, 1977, p. 14; 
Wise, 1976) and embodies the elements of worth but not necessarily 
merit. Excellence intimates an absolute superior standard of attain- 
ment, standards that are not time or context bound and are good in their 
ov/n right; therefore, the merit criteria are met but not necessarily 
the worth criteria. The concept of quality embodies characteristics 
of b'oth merit and worth; that is, a high (but not necessarily superior) 
level of attainment is required that also has utility or worth (makes 
sense and feels right) for those involved in the experience. 

For example, consider two programs designed to enhance the personal 
development of students in a residence hall. One activity is a series 
of workshops with the purpose of increasing interpersonal communication 
skills. The purpose of this program has both merit and worth. Liberally 
educated individuals are expected to be able to interact with a wide 
variety of persons (merit). Also students in the hall have exhibited 
a need for and expressed an interest in further developing their capacity 
to effectively communicate with their male and female counterparts (worth). 
The purpose of this program is defensible . 

The second activity's purpose is to develop autonomy among residents 
in anotnef hall. This purpose has merit ir> that the capacity "for autono- 
mous, independent thinking and decision makinq is valued /in an educated 
society. However, the activity ma£ not have worth if the residence unit is 
experiencing difficulty coordinating the policy making functions of its 
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various committees. An activity that encourages , at this particular 
point in time, active experimentation with behaviors characteristic 
of assertive or independent thinking persons ,may be ill-timed because 
such experimentation may actually impede attainment of the larger 
community's goals. Perhaps another set of activities addressing 
inter dependence among residents would be more appropriate and therefore 
of higher quality at that particular time with that particular 
group of students. 

Quality is a many-faceted property. While not "mystical" (Astin, 
1980), quality is difficult to assess. Nevertheless, judgments about 
quality are rendered everyday. Intra- and interinstitutional compari- 
sons fon the purposes of resource allocation are common (Astin, 1980; 
Jenny, 1979; Young, 1976). Some of the problems associated with measur- 
ing quality f may be exacerbated by flawed frameworks within which the 
elements contributing to quality have been considered. 

Frameworks for Assessing Quality 
Most of the conceptual frameworks^ available are essentially 
unidimensional assessment strategies which rely almost exclusively 
on quantitative indicators such as student ability or library re- , 
sources. For example, in a simplistic form of an input-output or 
"pipeline" approach, the guiding assumption is that college is 
essentially a conduit through which raw materials pass relatively 
untouched. If high ability students matriculate, the college can 
expect to have many highly able, successful alumni. In other words, 
'becaus-e the "impact" of the college is negligible, institutional 
quality is unimportant except insofar as attracting students is concerned. 



Most observers agree that multiple dimensions of an experience 
must be considered to estimate the degree to which quality, is present 
(Epstein & McParland, 1976; Palola, 1976; Sclmon, 1981). Perhaps the 
most popular multidimensional approach to assessing quality is the 
input-environment-output model (Astin, 1977, 1980). Inputs re- 
present what students bring with them to college (ability, interests, 
etc-); outcomes are measures of cognitive and affective changes be- 
tween students and others (faculty, peers, etc.) as well as actual 
institutional resources such as library holdings and soJ;orth. 

The.role of the college environment can be interpreted from 

two perspectives (Astin, 1977). As a "factory," the institution 

takes the raw materials (students' input characteristics), rearranges, 

reconstitutes,_or refines these materials, and produces a particular * 

kind of product (developed student). Astin (1977) suggested that . ; • 

a medical model perspective may have greater utility in understanding 

the influence of the environment on students' development. Of course, " * 

students are not considered "sick;" however, the extent to and ways in 

which students benefit from college is not unlike an ill patient with 

access to a medical facility. Lf the student plays an active role in 

the process, takes advantage of the opportunities available, and under- 

• « * 

stands how the experience will assist in attaining valued goals, the 

chances for a satisfactory learning experience ("health") increase. In 

this interpretation, the institution a*nd the student shar e responsibility 

for students 1 outcomes. 

The availability of certain institutional resources such as faculty, 

intramurals, library, and laboratory facilitie , and student support 



services and students 1 use of the s e resources make conceptually distinct 
contributions' to quality. The former are indicators and are comprised 
*of surrogate or proxy student or institutional characteristics. The 
latter are after-the-fact manifestations of student effort. To be 
useful in ^stimatino q-iality, indicators should be emoirically re- 
lated to manifestations of quality (Straumanis, 1981). 

A Comprehensive Quajity Assurance Framework 

1 Totnore clearly understand the relationship of qulaity to various* 
elements, of 'the undergraduate experience, the structured components of 
"a popular planning and evaluation model (Stufflebeam et al_, 1971) were 
redefined. The following conceptual framework resulted: 

(a) Context— characteristics of an institution's environment that 

% are relatively stable over time such as expenditures per student, size 

# of student body, and proportion of faculty with doctorates. 

« 

(b) Input— characteristics of entering students such as intellectual 
ability, interests, and aspirations. 

/(c) Involvement— interactions (manifestations) that characterize 
the environment in which students live and learn -such as the amount, 
type, and opportunity for informal interaction between students and 
faculty, students 1 satisfaction with tie institution, and student effort 
(Pace, 1980). 

(d) Outcomes -intended products or unintended effects (manifesta- 
tions) associated with college attendance such as retention rates, achieve- 
ment levels, student .development measures, and alumni achievements. 

Using these categories as advance organizers, the opinion and 
empirical research related to quality were reviewed and synthesized 

ii 
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to answer the following questions: 

(a) What kind of indices are available "to estimate quality? 

(b) What indices are used most often to estimate quality and 
how are they derived? 

(c) How can estimates of quality be improved? 

Review of the Literature on What Constitutes 
Quality in the Undergraduate Experience 

V 

Over= 200 literature entries were reviewed including: 

(a) Articles appearing in professional educational journals such 
as Change , Journal of Higher Education , Review of Educational Research , 
and monographs such as those produced in the ERIC/AAHL Research Report 

.series; * 

(b) Pertinent documents identified through a computerized search 
.conducted by the ERIC Clearinghouse on Higher Education; 

(c) Books and other published works reporting social and behavioral 
science efforts to estimate quality of life; 

(d) Previously unpublished material and reports such as papers 
predated at recent meetings such as the American Association of Higher 
Education, <the American Educational Research Association, and the Associa- 
tion for Institution Research. 

After a preliminary review, the materials were assigned to one or 
mane of the following categories for a more detailed analysis: (a) context 
indicators, (b) input indicators, (c) involvement manifestations, (d) 
outcome manifestations, and (e) quality assessment methodologies. Then 
the literature related to each of the categories was independently analyzed 
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and sunuarized. The c-iteria used to determine salient indices of quality 
were empirical support, cogency,. of argument, and repetition of mention 
(see Hutson, 1979 for a more detailed explanation of this literature re- 
view procedure). From this analysis, conclusions were drawn concerning: 
(a) whether various indices" of qual ity 'appear useful; (b) the degree to 
which institutional' agents such as administrators and faculty contribute 
to the quality of the undergraduate student experience, v 

In the following sections, salient indices of quality reported in 
the literature are sumnarized: A more detailed analysis of the relative 
utility of these indices in estimating the quality of the undergraduate 
experience can be found in Kuh (in press ). 

Context 

Four context indices may be worthwhile indicators of- qual ity at many 
institutions. Many positive indicators and manifestations of quality have 
been associated with institutional size (Bowa'n, 1977; Rock, Centra & Linn, 
1979). For example, smaller institutions tend to have clearer purposes; 
as a. result-, a greater sense of community is often fostered among students 
attending smaller institutions (Heath, 1968). In addition, opportunities 
for assuming leadership positions in cocurricular activities such as stu- 
dent government and informal contact with faculty tend to increase (Astin, 
1977; Chickering, 1969; Pace, 1981). Of course, the size of an institu- 
tion does pot cause, these things to happen; rather size encourages their 
occurence (Table 1). 

Clarity of institutional purpose has been referred to repeatedly as j 
an earmark "of a high quality institution (Astin, 1980; Meeth, 1974; Keetoni, 
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1971), Large universities must have multiple purposes ana missions to 
satisfy their diverse and sorhetimes competing stakeholding audiences 
(e,g,, trustees, taxpayers, students, alumni). Therefore, such insti- 
tutions rarely are able to project a salient purpose to most persons 
most of the time. 

Student living environments also appear functionally linked to 
quality (Astin, 1977; Chickering, 1974; DeCoster ft Mable, 1930; Feldman 
& Newcomb, .1969). Students spend a disproportionate amount of time 
during the undergraduate ynars engaged in nonclassroom related activities 
(Wilson, 1966)- Therefore, the degrees to which students are challenged 
by and satisfied with their living environments merit continued emphasis 
in the search for quality. 

Formal systematic properties such as administrative structures and 
decision making strategies have not been empirically related to quality 
(Bowen, 1979). However, the informal organization comprised of norms 
that encourage or hinder faculty involvement with student: and the extent 
to which students feel and act as though they are members of the academic 
community seem to be particularly promising areas of inquiry for those 
interested in enhancing the quality of the undergraduate experience (Benezet 
1981). 



Insert Table 1 about here* 



Input ^ 
Although input indicators such as high school class rank and SAT or 
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ACT scores have often been used (Bowen, 1981; Millett, 1979), few empirical 
relationships between manifestations of quality such as value-added achieve- 
ment or retention rates and input indicators have been found (Astin,, 1977), 
The degree to which nonintellective characteristics of students such as 
interests and aspirations is consistent with an institution's environmental 
press has been positively correlated with student satisfaction and persis- 
tence (Chickering, 1974; Moos, 1979; Pace, 1969; Stern, 1970). But as 
quality indicators, these variables have not been powerful predictors of 
quality manifestations (Table 2). 

. Student ability is perhaps the most often used indicator of quality 
(Astin & Solmon, 1979). Yet, various inquiries with slightly different 
foci have been unable to document whether the ability of students is 
positively related to the quality of the experience. Most studies linking 
ability with subsequent manifestations of quality have been quantitative. 

r 

J i 

Perhaps case study portrayals of students with varying levels of ability 
in institutions with different purposes could help fill the apparent 
knowledge void about the relationship between student ability and quality' 
in the undergraduate experience. 

Insert Table 2 about here 

In volvement 

The degree to which students are involved during the undergraduate 
years is one of the most important and, perhaps, accurate manifestations 
of quality (Astin, 1977, 1979; Astin & Scherrei , 1930; Pace, 1980; Scott, 
1980). Involvement is related to a variety of other indicators and mani- 



festations often empirically or logically associated with quality: in- 
stitutional size, general satisfaction with the institution (including 
satisfaction with the living environment and academic pursuits --see 
Kegan, 1978), persistence to completion of degree objective (Astin, 
1977), postcollege community service, and income* The "quality" of 
effort students expend is currently receiving attention (Pace, 1979, 
1980, 1981) and may prove helpful in distinguishing between students 
benefiting in profound ways from college and their counterparts who be- 
come dissatisfied and/or leave the institution (Table 3). 

The degree to which faculty expend effort in instruction or are 
involved with students out of the classroom has been positively related 
to many manifej^ations of quality (Bean, 1981; Bragg, 1976; McKeachie, 
1978; Menges, 1981; Pascarella, 1980; Sanford, 1967; Snyder, 1971; Stu- 
dent Task Force on Education at Stanford, 1973, Trow, 1975). Identifica- 
tion of the factors related to faculty involvement (e.g., morale, 
salience of institutional purpose) seems a promising line of inquiry. 



Insert Table 3 about here 



Outcomes 

Persistence enables J^udents to take advantage of a variety of other 
opportunities related to quality (Astin, 1979); e.g., interaction with 
faculty and peers and participation or perhaps leadership in institutional 
governing processes (Table 4). Therefore, persistence is, like size, a 
mediating index>. Yet persistence may have a pernicious influence as well 
(Chickering, 1971). Some students may become placated or too satisfied 

* 
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with the college experience. A number of these students probably per- 
sist to graduation but the meaninqs they make of their expriences have 
been rarely challenged, especially in their livinq environments. This is 
more likely to be true for the increasing proportions of .part-time cotomut 
ing students (Chickering, 1974; Astin, 1977). 

A'lumni studies are thought to be perhaps the best source of infor- 
mation about college quality (Boulding, 1975; Freedman, 1962; Pace, 1979) 
Bowen (1979) suggested that many important outcomes of college cannot 
be documented at commencement because the impacts or changes will not 
be manifested until some, years later.' 

The residue of a college education— after the 
initial forgetting of detail—is a virtual mystery. 1 
Moreover, we should be interested in the values and 
1 attitudes of alumni, their interests, their citizen- 
ship, their family life, arid their careers as these 
may have been affected by their college experiences 
(P. 25).. 

The search for value added outcomes of college have met with little 
success (Astin & Panos, 1979). Aggregation and measurement of change 
problems have often been cited as contributing factors to insignificant 
findings (Astin, 1977). Interestingly, those -few researchers (e.g., 
Cottle, 1975; Heath, 1968; Perry, 1970; White, 1966) who have conducted 
intensive studies of small samples of students atid alumnj have reported 
provocative accouts of what happens to students during and after college. 
Like quantitative assessments of the same phenomena, these approaches 
also have limitations; but the insights .they may provide concerning value 
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added outcomes of college are descriptively rich (see Trow, 1975). 

The empirical evidence has suggested that graduating from a higher 
"quality" college will result in greater earnings during the post-college 
years (Solmon, 1972; Wachtell , 1975) . It is unclear whether income is 
more a function of the contributions of the undergraduate experience or 
some other variable such as socioeconomic status (see Karabel & Astin, 
1972) which may be unrelated to the quality of the education experience. 

Insert Table 4 about here 



Quantitative and Qualitative Contributions to Quality Assessment 
Quality is a multidimensional concept. After reviewing the literature 
on quality in the undergraduate experience, it is clearer why: (1) few 
assessment efforts which attempt to account for the multiple properties 
of quality have been mounted; and (2) most "quality assessments' 1 rely 
on indicators rather than* manifestations of quality. 

In general, methodological approaches to quality assessment can be 
divided into two categories: quantitative and qualitative. Each represents 
a different way of attempting to describe the same series of events. In 
practice, most intrainstitutional efforts to estimate quality incorporate 
elements from both paradinms. These two approaches are separated for dis- 
cussion purposes for two reasons: (1) adherence to one approach to the 
exclusion of the other substantially influences (a) what is assessed and 
(b) the assessment startegies employed; (2) quantitative methods have 
dominated quality assessments for a considerable period of time and therefore 
it seems appropriate to illuminate the advantages, limitations, and relative 
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utility of different' approaches (Stark & Lowther, 1980). Guba's (1978) 
points of comparison between conventional and naturalistic inquiry serve 
as a framework within which the contributions of different approaches 
to estimating quality can be considered. 

Quantitative 

Quality assessments are often based in- the logical positivism 
characteristic of the agricultural-botbny and psychometric traditions 
(Sherman, 1981) and rely almost excXMsively on operationally defined 
and objective measures .(Astin & Solmon, 1979). Therefore, quantitative 
assessments usually' focus on institutional factors about which objective, 
standardized measures are availa le or can be collected. A pVedetermined, 
reductionistic framework is used to identify those variables which 
appear to be related to the factor of interest. 

In a sense, the quality assessment is almost unrelated to human inter- 
actions and tends to overlook institutional conditions that cannot be 
"technically 11 controlled (i.e. through conventional inquiry methods such 
as inferential statistics). The only value perspective allowed to in- 
fluence the assessment is that of the investiqatar's (see Table 5), 

Quantitative approaches are attractive because they produce data 
which: (1) can be used for both intra- and interinstitutionat comparisons; 
(2) are amenable to computer assisted analytic procedures; and (3) are 
compatible with the psychometric paradigm which has influenced the social 
sciences for decades. However, quantitative measures are less likely to 
account for unintended outcomes and acknowledge the range and depth of 
human experiences. While multivariate procedures are available, most 

s 
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quantitative assessments have emphasized unidimensional perspectives on 
quality. Quantitative assessments of quality often correlate two or more 
input, context, or outcome variables such as aptitude or achievement exam 
scores such as Scholastic Aptitude Test or Graduate Record Examination ; 
high school class rank; expenditures per student; proportion of faculty 
with doctorates, faculty salaries, library collections, and number of" 
fellowships awarded to seniors. 




Insert Table 5 about- here 



Qualitative 

Sometimes referred to as "naturalistic" or ethnographic, qualitative 
assessments value an expansionist, phenomenological perspective compatible 
with ethnography. The guiding assumption is there are different ways of 
knowing and making meaning. Therefore, to adequately estimate the quality 
of the experience, those invloved in the process (students, faculty, staff) 
must be giveh "an opportunity to describe. . .and comment on the meaning of 
those experiences for them" (Wolf, 1979, p. 1). Rather than indicators, 
manifestations such as students 1 reports of satisfaction with various as- 
pects of the institution or observations of students' involvement in both 
in- and out-or-class activities are primary data sources. Qualitative, 
assessments embrace a variety of relatively "subjective" data collection 
strategies such as observations, interviews, case studies, and review of 
existing documents (Sherman, 1981). The da*;a collected are "conf irmable" 
(Guba, 1978), but reflect the pluralistic values embedded in the institu- 
tional context. Graphic portrayals (what Guba refers to as "thick") result 
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which communicate a sense of "what" has been experienced "how". 

Qualitative assessments are sensitive to unintended effects and outcomes. 
However, they tend to be labor intensive, appear less objective, and are 
generally cumbersome when used for interinstitutional quality assessments. 
The -advantages and disadvantages of quantitative and qualitative approaches 
to quality assessment are summarized in Fugure 1. 

Toward a "Holistic" Appraisal of Quality 
Can quality be adequately estimated using either a quantitative or 
qualitative assessment technique? Because quality is a multidimensional 
concept, data gathering strategies characteristic of both approaches are 
required. Exclusive use of one or the other wil", likely overlook elements 
of the undergraduate experience that could markedly influence conclusions 
about quality. More important, to determine what can be done to improve 
the present level of quality will require multiple perspectives on the 
college expedience (Astin, 1980). For example, while quantitative efforts 
to rank institutions by the "quality" (ability) of their students make in- 
teresting reading (r^e Astin .& Solmon, 1979), they offer little assistance 
to faculty and staff charged with identifying programs or institutional 
policies that could be^modified to enhance quality. 

A holistic, eclectic perspective of quality assessment recognizes the 
validity and importance of as well as the limitations of contrasting inquiry 
methods and the different kinds and sources of information emphasized in , 
the quantitative and qualitative approaches (Stark & Lowther, 1980). In 
holistic quality assessments the data gathering strategies are eclectic 
and emergent in that the methods employed are dictated by students' purposes 
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and characteristics, the institution's mission, an> the purposes for the 
quality assessment. Usually more than one Torm of data gathering are re- 
quired to illuminate the various aspects of the students' experience (see 
Parlett and Hamilton, 1976 for a detailed discussion of holistic "illumi- 
native" strategies). 

Guiding Principles for Holistic Quality Assessments 

Whan considered together, the six principles introduced .below se>*ve 

as a general framework for holistic quality assessment. Of course, certain 

'institutional factors or circumstances may militate against the use of one 

or more. In general, however, these guideposts will lend direction to 

quality assessment activities. 

1 • Quality assessment is guided by the purpose(s) of the target 
jpj^anior_unit an d of the students participating in target- 
sponsored activities. 

One of the earmarks of quality often mentioned in the literature is 

purpose. Therefore, the degree to which the activities of t u e program or 

J 

unit under study are compatable with the stated purposes of both the target 
program and the students may be an indicator of quality. The guiding ques- 
tion is, "What are we trying to do hare together?" 

Focusing on institutional purposes will probably increase awareness 
to the "problematic preferences" (competing and sometimes conflicting in- 
stitutional goals— see Cohen & March, 1972) common to larger institutions.. 
Therefore, the scope of the assessment should be consistent with levels of 
the organization with the clearest purposes. In other words, focusing quality 
assessment efforts at the academic program "ievel within a college or 
on students' -living units are more likely Vo prove helpful than .attempting 
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to estimate the "quality" of the entire institution. 

2. Quality assessment seeks ma n ifestations or information about 
involvement in and outcomes >f the college experience . 

Using indicators i:o estimate quality is tempting because indicator 
data are usually readily available and are relatively easy to manipulate. 
Quality in the undergraduate experience is not preordinate, however* 
Rather, quality results from the interaction between students and institu- 
tional agents or others (e.g., peers). Therefore, questions of interest 
to quality assessers are "What do students do here (involvement) ?" and 
"What happens to students as a result' of their involvement (outcomes)?" 
To adequately respond to the latter question, assessment activities must 
extend beyond the campus borders and into the post-college years, 

3. Quality assessment is action oriented . 

An important objective of "quality assessment is to inform arid guide 
the implementation of interventions designed to enhance the quality of 
students 1 experiences. That is, ""What actions do our experiences collecting 
the information and the data thtmselves suggest concerning how to enhance <■ 
quality?" Gathering information for purposes of interinstitutional comparison 
may have merit. However, more persons in an institution are likely to 
benefit in more ways if the assessers remain sensitive to changes in policies 
^and practices which could increase quality. In this senso, quality assess- 
ment is both descriptive and evaluative. 

4. Quality assessment requires multiple forms of d^ta gathering . 
"What methods and strategies will't&ll us what we need to" know about 

the quality of students 1 experiences?" Without different ways of collecting 
information about students 1 and faculty behavior and satisfaction with their 
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interactions, the multiple realities that exist will not be adequately 
recorded.. Triangulation, cross validation, and a combination of quali- 
tative and quantitative data are required if a reasonably accurate, esti- 
mate of the quality of the undergraduate experience is to result. 

5 - Quality assessment is a, public, conscious, and educative activity . 

To-become familiar with the purposes and expectations of students as 
well as institutional agents, quality assessment activities will generate 
more useful information when the requirements and perspectives of various ' 
stakeholder groups are considered. To operationalize this principle, 
collaboration between "direct" (student, faculty, staff) and "indirect" 
(alumni, trustees, legislature, others) contributors to quality is required 
to adequately describe the quality of the undergraduate experience. "How 
can -various constituent groups be encouraged to participate and benefit 
from quality assurance activities?" 

Participation will enable students and institutional agents to learn 
more about their respective roles and to reflect on the meaning of their 
experiences. Access to the process will also maintain enthusiasm for 
and committment to acting on information generated from the assessment. 
Assessments will have greater internal credibility if dgring and after 
the process, participants know more about the program or unit, themselves, 
and what connotes quality than they did at the outset. Stakeholders 
should be periodically informed of the 'assessment strategy, findings, 
and potential consequences of possible quality enhancing interventions 
as they are identified; Particular benefits may accrue to students who 
experience the complexities of translating findings into action-based 
alternatives. 

24 
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Quality assessment is value laden . 

— 1 : b 

Pluralistic value orientations of students* faculty across units, 

staff, and indirect contributors make it difficult if not impossible to 

achieve consensus* concerning what quality is and how quality can be 

enhanced. Both the*meaning and measurement of quality are tied to local 

values; therefore, objective or "scentif ic" approaches to quality assur- 

ance nay be' flawed from thPstart. "What is important to the people in 

this place?" The smaller the unit and numbers of persons involved, the 

easier it is to respond and share understandings of value laden issues. 

Collaboration and negotiation may not satisfactorily resolve disagreements. 

But t&assume quality is not tied" to values is to* ignore the differences 

between the quality- of students' experiences at one institution and stu- 

V 

dents 1 experiences at another. 
* Conclusion * 

Institutional agents can assess o\ adequately -estimate the degree 
to which quality is present. However, quality assessment is a rather 
complex process which requires "muddling through" and a high degree of 
collaboration. Estimates of quality used for reasons other than intra- 
institutional policy decisions may be suspect, given the context-relative 

nature of "quality". • . 

» * 
To assess quality, emphasis must be given to the relationship between 1 

the purpose(s) of an activity or program and the manifestations or during- 

and after-the-fact evidence that the college experience tvas meaningful and 

resulted in "value added" changes on * the part of students and society. 

Persons 1 values differ as'do the meanings they attach to the same experience* 

Multiple .perspectives on the quality ol^undergraduate experience must be 

acknowledged and can be adequately estimated only through the use of- 

♦ 

different but complimentary data gathering approaches.. 
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Table 1 34 

Relationships Between Quality and Selected Context Variables 



Independent 
Variable 



Dependent 
Variable 



Direction and Strength 
of Relationship 



Institution size 
Rock Centra & Linn (1970) 
Astin (196&) 
Astin (1977) 

• Astin (1968b) 

Chickering (1969); Heath (1968) 

* Bowen (1977) 
Me^th (1974) 

Department size 
Millett (1979) 
Meeth (1974) 

Institutional .purpose ♦ 
Astin (1980); Chickering (1969); Heath 
(1968); Keeton (1971); Meeth (1974) 



-''moderate to low 
0 



Achievement 
Achievement 
Campus leadership - moderate 

Perceived prestige - moderate 

Personal development 

opportunities 
"Educational advantage" 
Cost per student 0 



Department "quality" + 
Faculty /student interaction + 



Developmental ly powerful 
community 



Organizational properties 
Troutt (1979) 

, BeSn (1981); Benezet (1981); Berdie(1972) Students involvement in 

and understanding of 



Accreditation 



- moderate to hiqh 



0 
+ 



Bowen (1979) 

Br^gg (1976); Gaff & Gaff (1981); 
Stanford Student Task Force (1973) 

Grading practices 
Wilson- (1969) 

Financial resources 
Troutt (1979) 
Astin (1968a) 

Rock, Centra & Linn (1970) 
Adams & Krislov (1978) 
Solmon (1972, 1975) 
. Bowen (1981); Meeth (1974). 

Student living environments 
Wilson (1966) 

Astin (1977); Chickering (1974); 

FeTdman & Newcomb (1969) 
Williams, Riley & Zgliczynski -(1980) 



institutional operations 

"Quality" 0 
Student/faculty interaction + 



Achievement (GPA) 



Accreditation + 

GRE scores 0 

GRE scores + low 

Gourman (1977) Index + moderate 

Post college income + low 

"Quality" 0 

Knowledge acquisition + 

Personal development 0,+ low 

Grades, persistence and + 
involvement 
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Table 2 

Relationships between Quality and Selected Input Variables 





Variable 


npnpnHpnf' ^ 

uCpCIIUCII w 

Variable 


of Relationship 


Ability • 


****** 

- 




Astin (1968); Astin & 

Panos (1972) 

* 


Perceived institutional 
affluence 


+' "moderate 

» 


Non-intellective/biographical 
characteristics 


- 


- • 


•Willingham (1980) 


Liberating experience 


+„ low 


Moos (1979); Stern (1970) 

r 

Astin (1968) 


Satisfaction with 
institution 


+ low 


Student-environment fit 


+ low 

* 
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Table 3 

Relationships between Quality and Selected Involvement Variables 
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Independent 
VariaWe • 



-Dependent 
Variable • 



Peer interaction 
Astin & Scherrei (1980); Feldman & 

Newcomb .(1969) 
Astin (1977) 



Kegan (1978) 

Chickering (1974); Brkag (1976) 



M9 

Newcomb (1962); SternTT962) 



Instruction 
Menges (1981); McKeachie (1978) 
Kolb (1981); Messick et al. (1976) 

Informal student-facul ty interaction 
Pascarella (1980) 

Stanford Task Force (1973); 

Sanford (1967); Trow (1975) 
Centra & Rock O970) 
8ean (1981) 
Thistlewaite (1959) 
Bragg (1976); Feldman & 

Newcomb (1969) 

Student effort 
Brown (1937) 

Astin (1980) 

Pace (1979, 1980,^1981) # 



"Quality" 

Persistence, personal, 
development, satis- 
faction 1 

Satisfaction 

Personal development 

Attrition, dissatis-* 

faction - . 



Direction and Strength 
of Relationship g 



+ low-moderate 
' + low-moderate 

t 

+ moderate 



Learning outcomes + 
Development of intellect + 



Student development; + 

satisfaction 

"Quality" + 

GRE scores + moderate 

Persistence + 

Graduate study + 

Socialization + 



Maximizing institutional + 

resources , 
Achievement t 
Achievement, personal + 

development 
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' Table 4 

Relationships .Between Quality and Selected Outcome Variables 
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1 _ ^ 


V 

Independent 
Vaniable 


Dependent 
Variable 


Direction and Strength 
of Relationship ' 


•Persistence 

Astin (1979); Kenan (1974) 
Pascarella (1980); Pascarella & 
Terenzini- (1977) ; Rossmann 
(1967); Spady (1971) 
> Astin (1975); Blake (1971); 
Seibel (1973) * 
Astin (1975) 
Astin (1975, 1977) 
' > . . * • 


General satisfaction 
Faculty-student inter- 
action 

Peer involvement 

Se,lf esteem 

Grade point average 


• + moderate 

+ moderate 
-+ low-moderate 


Achievement 
Astin & Panos (1969) 

Pascarella (1980) 

Sharp (T970) * 


"Value added" achieve- 
ment 

Faculty-student inter- 
action 

Graduate school, oc- 
cupation 


• 

o- 

+ moderate 
0 


Personal development 
flickering, McDowell & 
Campagna (1969); Clark et al. 
(1972); Trent I Medsker '(1969) 


Institutional type 


0 . . 

• 


Post-college salary 
- Solmon (1975)- • 
Bowen (1977) 


Gourman index 
"Quality" college 


+ 
+ 


Alumni activities 
Spaeth &-Greely (1970) 


Institutional "quality" 


0 
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Table 5 

Advantages and Disadvantages of Quantitative 
and Qualitative Approaches to Quality Assessment 



Quantitative 



Qualitative 



Advani 



ages: 



Reliance on psychometric inquiry. 
' paradigm (objective measures.) 

Preordinate design results in 
greater efficiency 

Data amenable to traditional 

computer driven analytic 
r procedures 

Data amenable to intra- and 
* interinstitutional comparisons 
across time (generalizability) 



Disadvantages: 

Underemphasize involvement 
manifestations of quality 

Relatively insensitive to un- 
intended effects or outcomes 
(serendipity) 

tridimensional (single reality)^ 
inquiry is empfuisized 

Diversity in students' experi- 
* ences is masked 



Advantages: * 

.Acknowledges multiple realities and 
purposes . 

Seeks context and participant relevant 
data 

Provides rich, "thick" descriptions 
of students 1 experiences 

Serendipity is documented. 

Responsive to emergent issues and ' 

pluralistic values/ ; perspgctives related 
• to quality of student experiences 

Disadvantages: 

Labor intensive 

Data not easily ^collated or compared 
within or* across institutions *. 

i 

Ipput and^ to a lesser extent, context 
indicators are deemphasized 

Results seem "subjective", not "hard" 
evi derate 



/ 
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